24 



WHAT IS CLAIMED IS: 

1 . A method for deriving a three-dimensional panorama from 
a plurality of images of a scene generated by a range imaging camera of the type 
that produces ambiguities in range information, said method comprising the steps 
of: 

(a) acquiring a plurality of adjacent images of the scene, wherein 
there is an overlap region between the adjacent images and at least some of the 
adjacent images are range images; 

(b) providing offset data for the range images in order to recover 
corrected relative scene spatial information, wherein the step of providing offset 
data further comprises: 

(i) detecting a relative range difference between adjacent 
range images as a constant offset between the adjacent images; 

(ii) applying the constant offset to at least one of adjacent 
range images to correct for ambiguities in the relative ranges of the range images, 
thereby providing corrected range images; and 

(c) deriving a three-dimensional panorama from the corrected 

range images. 

2. The method as claimed in claim 1 wherein scene spatial 
information is provided as image values in a local three-dimensional coordinate 
system of each of the images and the step (c) of deriving a three-dimensional 
panorama comprises the steps of: 

(i) transforming the image values from each of the local three- 
dimensional coordinate systems of each of the images to a selected reference 
three-dimensional world coordinate system, thereby providing transformed range 
images; 

(ii) warping the transformed range images onto a cylindrical 
surface, and forming a plurality of warped range images; 

(iii) registering adjacent warped range images; and 
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(iv) deriving the three-dimensional panorama using the warped 

range images. 

3. The method as claimed in claim 1 wherein the step (i) of 
detecting a relative range difference between adjacent range images as a constant 
offset comprises the steps of: 

(11) predicting the relative range difference by an estimated 
constant offset; 

(12) warping the range images onto a cylindrical surface using the 
estimated constant offset, and forming a plurality of warped range images; 

(13) registering adjacent warped range images, thereby producing 
predicted range values; 

(14) evaluating any error between the predicted range values and 
the actual range values in the overlap region; 

(15) if the error is unacceptable, using an optimization routine to 
select another estimated constant offset; and 

(16) repeating the steps (i2) through (i5) until the error is 
acceptable, thereby producing the constant offset. 

4. A method for deriving a three-dimensional panorama from 
a plurality of images of a scene generated from a range imaging camera of the 
type that produces ambiguities in range information, said method comprising the 
steps of: 

(a) acquiring a plurality of images of the scene by rotating the 
camera about a Y-axis (vertical axis), wherein there is an inter-overlap region 
between adjacent images; 

(b) providing offset data for each image to recover corrected 
relative scene spatial information (X,Y,Z) with respect to a local XYZ coordinate 
system; 

(c) selecting a reference three-dimensional world coordinate 
system against which spatial information of the scene can be correctly presented; 
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(d) transforming the corrected relative scene spatial information 
(X,Y,Z) from each of the local three-dimensional coordinate systems of each of 
the images to the selected reference three-dimensional world coordinate system, 
thereby providing transformed (X,Y,Z) images; 

(e) warping the transformed (X,Y,Z) images onto a cylindrical 
surface, and forming a plurality of warped (X,Y,Z) images; 

(f) registering adjacent warped (X,Y,Z) images; and 

(g) forming a three-dimensional (X,Y,Z) panorama using the 
warped (X,Y,Z) images. 

5. The method claimed in claim 4, wherein the plurality of 
images includes range images and the step (b) of providing offset data further 
comprises the steps of: 

(i) detecting differences in constant offset between the range 

images; and 

(ii) using the differences to correct for ambiguities between the 

range images. 

6. The method as claimed in claim 4 wherein the plurality of 
images generated from the range imaging camera includes color images and the 
three dimensional panorama is in color. 

7. The method claimed in claim 4 wherein the reference three- 
dimensional world coordinate system is an arbitrary three-dimensional coordinate 
system. 

8. The method claimed in claim 7 further comprising the step 
of selecting the reference three-dimensional world coordinate system from the 
local three-dimensional coordinate systems or a three-dimensional coordinate 
system defined elsewhere. 
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9. The method claimed in claim 4 wherein step (d) of 
transforming the corrected relative scene spatial information (X,Y,Z) comprises 
forming a homogeneous transformation matrix. 

10. The method as claimed in claim 4 wherein each image is 
captured as a bundle of associated images, said bundle including a plurality of 
phase images each incorporating the effect of a predetermined modulation 
frequency together with a phase offset unique for each image. 

1 1 . The method as claimed in claim 1 0 wherein each range 
image is generated from a respective plurality of phase images associated with 
each bundle. 

12. The method as claimed in claim 10 wherein the bundle also 
includes an intensity image. 

1 3 . The method as claimed in claim 1 2 wherein the intensity 
image is a color image. 

14. A three-dimensional panoramic system for producing a 
sequence of spatial (X,Y,Z) images and a sequence of intensity (R,G,B) images, 
comprising: 

(a) a panoramic three-dimensional scannerless range imaging 
capture component for acquiring a plurality of image bundles of the scene, 
wherein each image bundle includes an intensity (R,G,B) image and a plurality of 
phase images from which a spatial (X,Y,Z) image is derived, and wherein there is 
an overlap region between adjacent image bundles; 

(b) a reference coordinate component for warping image pixels 
from each image bundle onto a cylindrical surface, thereby generating a plurality 
of warped images, said reference coordinate component including means for 
registering one or more common image pixels in the overlap regions of adjacent 
warped images, thereby providing a plurality of registered warped images; 
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(c) an image stitching component for stitching the overlap regions 
of the registered warped images to generate a panorama; and 

(d) a graphics display for visually displaying the panorama. 

15. A three-dimensional panoramic imaging system, 

comprising: 

(a) a three-dimensional panoramic capturing system, wherein a 
sequence of spatial (X,Y,Z) images and a sequence of intensity (R,G,B) images 
are produced; 

(b) a reference coordinate transformation system for generating 
transformed spatial images, said transformation system comprising a general 
homogenous transformation matrix for transforming each of the spatial images 
into a common three-dimensional coordinate system from its local three- 
dimensional coordinate system at which the corresponding intensity image is 
taken and the original spatial image is computed; 

(c) an image stitching system that produces a stitched spatial 
panorama from the transformed spatial images, and a stitched intensity panorama 
from the sequence of intensity images; and 

(d) a graphics display system for receiving the stitched spatial and 
intensity panoramas and generating a virtual world reality. 

16. A computer program product for deriving a three- 
dimensional panorama from a plurality of images of a scene generated by a range 
imaging camera of the type that produces ambiguities in range information, said 
computer program product comprising: a computer readable storage medium 
having a computer program stored thereon for performing the steps of: 

(a) accessing a plurality of adjacent images of the scene, wherein 
there is an overlap region between the adjacent images and at least some of the 
adjacent images are range images; 

(b) providing offset data for the range images in order to recover 
corrected relative scene spatial information, wherein the step of providing offset 
data further comprises: 
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(i) detecting a relative range difference between adjacent 
range images as a constant offset between the adjacent images; 

(ii) applying the constant offset to at least one of adjacent 
range images to correct for ambiguities in the relative ranges of the range images, 
thereby providing corrected range images; and 

(c) deriving a three-dimensional panorama from the corrected 

range images. 

17. A computer program product for deriving a three- 
dimensional panorama from a plurality of images of a scene generated from a 
range imaging camera of the type that produces ambiguities in range information, 
said computer program product comprising: a computer readable storage medium 
having a computer program stored thereon for performing the steps of: 

(a) accessing a plurality of images of the scene by rotating the 
camera about a Y-axis (vertical axis), wherein there is an inter-overlap region 
between adjacent images; 

(b) providing offset data for each image to recover corrected 
relative scene spatial information (X, Y,Z) with respect to a local XYZ coordinate 
system; 

(c) selecting a reference three-dimensional world coordinate 
system against which spatial information of the scene can be correctly presented; 

(d) transforming the corrected relative scene spatial information 
(X,Y,Z) from each of the local three-dimensional coordinate systems of each of 
the images to the selected reference three-dimensional world coordinate system, 
thereby providing transformed (X,Y,Z) images; 

(e) warping the transformed (X, Y,Z) images onto a cylindrical 
surface, and forming a plurality of warped (X,Y,Z) images; 

(f) registering adjacent warped (X,Y,Z) images; and 

(g) forming a three-dimensional (X, Y,Z) panorama using the 
warped (X,Y,Z) images. 
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18. A computer program product as claimed in claim 17, 
wherein the plurality of images includes range images and the step (b) of 
providing offset data further comprises the steps of: 

(i) detecting differences in constant offset between the range 

images; and 

(ii) using the differences to correct for ambiguities between the 

range images. 



19. A computer program product as claimed in claim 12 
wherein step (d) of transforming the generated (X,Y,Z) values comprises forming 
a homogeneous transformation matrix. 



